{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LogisticRegression and Gradient desent\n",
    "Logistic回归是众多分类算法中的一员。通常，Logistic回归用于二分类问题，例如预测明天是否会下雨，当然它也可以用于多分类问题。\n",
    "\n",
    "假设现在有一些数据点，我们利用一条直线对这些点进行拟合(该线称为最佳拟合直线)，这个拟合过程就称作为回归，如下图所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](pic/16.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Logistic回归是分类方法，它利用的是Sigmoid函数阈值在[0,1]这个特性。Logistic回归进行分类的主要思想是：根据现有数据对分类边界线建立回归公式，以此进行分类。其实，Logistic本质上是一个基于条件概率的判别模型(Discriminative Model)。\n",
    "\n",
    "所以要想了解Logistic回归，我们必须先看一看Sigmoid函数 ，我们也可以称它为Logistic函数。它的公式如下："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](pic/18.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](pic/17.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](pic/19.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Sigmoid:\n",
    "\n",
    "\n",
    "\n",
    "![title](pic/20.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "z是一个矩阵，θ是参数列向量(要求解的)，x是样本列向量(给定的数据集)。θ^T表示θ的转置。g(z)函数实现了任意实数到[0,1]的映射，这样我们的数据集([x0,x1,...,xn])，不管是大于1或者小于0，都可以映射到[0,1]区间进行分类。hθ(x)给出了输出为1的概率。比如当hθ(x)=0.7，那么说明有70%的概率输出为1。输出为0的概率是输出为1的补集，也就是30%。\n",
    "\n",
    "如果我们有合适的参数列向量θ([θ0,θ1,...θn]^T)，以及样本列向量x([x0,x1,...,xn])，那么我们对样本x分类就可以通过上述公式计算出一个概率，如果这个概率大于0.5，我们就可以说样本是正样本，否则样本是负样本。\n",
    "\n",
    "举个例子，对于\"垃圾邮件判别问题\"，对于给定的邮件(样本)，我们定义非垃圾邮件为正类，垃圾邮件为负类。我们通过计算出的概率值即可判定邮件是否是垃圾邮件。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 如何得到合适的参数向量θ？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据sigmoid函数的特性，我们可以做出如下的假设：\n",
    "\n",
    "![title](pic/21.jpg)\n",
    "\n",
    "式即为在已知样本x和参数θ的情况下，样本x属性正样本(y=1)和负样本(y=0)的条件概率。理想状态下，根据上述公式，求出各个点的概率均为1，也就是完全分类都正确。但是考虑到实际情况，样本点的概率越接近于1，其分类效果越好。比如一个样本属于正样本的概率为0.51，那么我们就可以说明这个样本属于正样本。另一个样本属于正样本的概率为0.99，那么我们也可以说明这个样本属于正样本。但是显然，第二个样本概率更高，更具说服力。我们可以把上述两个概率公式合二为一：\n",
    "![title](pic/22.png)\n",
    "合并出来的Cost，我们称之为代价函数(Cost Function)。当y等于1时，(1-y)项(第二项)为0；当y等于0时，y项(第一项)为0。为了简化问题，我们对整个表达式求对数，(将指数问题对数化是处理数学问题常见的方法)：\n",
    "![title](pic/23.png)\n",
    "\n",
    "其中，m为样本的总数，y(i)表示第i个样本的类别，x(i)表示第i个样本，需要注意的是θ是多维向量，x(i)也是多维向量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "综上所述，满足J(θ)的最大的θ值即是我们需要求解的模型。\n",
    "\n",
    "怎么求解使J(θ)最大的θ值呢？因为是求最大值，所以我们需要使用梯度上升算法。如果面对的问题是求解使J(θ)最小的θ值，那么我们就需要使用梯度下降算法。面对我们这个问题，如果使J(θ) := -J(θ)，那么问题就从求极大值转换成求极小值了，使用的算法就从梯度上升算法变成了梯度下降算法，它们的思想都是相同的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 梯度上升:\n",
    "\n",
    "![title](pic/24.png)\n",
    "\n",
    "图像：\n",
    "![title](pic/25.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as  np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loadDataSet(path='data/testSet.txt'):  #读取测试文件\n",
    "    with open(path) as fr:\n",
    "        dataMat = [];\n",
    "        lableMat = []\n",
    "        for line in fr.readlines():   #处理data label\n",
    "            lineArr = line.strip().split()\n",
    "            dataMat.append([1.0, float(lineArr[0]), float(lineArr[1])])\n",
    "            lableMat.append(int(lineArr[-1]))\n",
    "        return dataMat, lableMat\n",
    "    return -1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataMat,labelMat=loadDataSet()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(inX):\n",
    "    return 1 / (1 + np.exp(-inX))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradAscent(dataMatIn, classLabels):   #梯度上升\n",
    "    dataMatrix = np.mat(dataMatIn)\n",
    "    labelMat = np.mat(classLabels).reshape(-1, 1)\n",
    "    m, n = np.shape(dataMatrix)\n",
    "    weight = np.ones((n, 1))\n",
    "    alpha = 0.001\n",
    "    for i in range(500):\n",
    "        h = sigmoid(dataMatrix * weight)\n",
    "        error = labelMat - h\n",
    "        weight = weight + alpha * dataMatrix.T * error  #这里是根据costfuc推导出来的公式\n",
    "    return weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plotBestFit(wei):   #画图\n",
    "    weights=np.array(wei)\n",
    "    dataMat, labelMat = loadDataSet()\n",
    "    dataArr = np.array(dataMat)\n",
    "    n = np.shape(dataArr)[0]\n",
    "    xcord1 = [];\n",
    "    ycord1 = []\n",
    "    xcord2 = [];\n",
    "    ycord2 = []\n",
    "    for i in range(n):\n",
    "        if int(labelMat[i]) == 1:\n",
    "            xcord1.append(dataArr[i, 1]);\n",
    "            ycord1.append(dataArr[i, 2])\n",
    "        else:\n",
    "            xcord2.append(dataArr[i, 1]);\n",
    "            ycord2.append(dataArr[i, 2])\n",
    "    fig = plt.figure()\n",
    "    ax = fig.add_subplot(111)\n",
    "    ax.scatter(xcord1, ycord1, s=30, c='red', marker='s')\n",
    "    ax.scatter(xcord2, ycord2, s=30, c='green')\n",
    "    x = np.arange(-3.0, 3.0, 0.1)\n",
    "    y = (-weights[0] - weights[1] * x.reshape(-1,1)) / weights[2]\n",
    "    ax.plot(x, y)\n",
    "    plt.xlabel('X1');\n",
    "    plt.ylabel('X2');\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de3SddZ3v8fc3Ta9JWnpJgLYppQmC3IrS0zrDoLBQbPGCrqWz4ByUGZVSRxmdM57q6FLn4Jo1WsfRs2CUFmHpjBX1HAVZUhTUcYGMhLbYUrSgabkkbSVpSy/pPcn3/LGfpDs7ez/ZSfZ+Lnt/XmtlNfvZT5Lvzk6f7/O7fX/m7oiIiBRSE3cAIiKSbEoUIiISSolCRERCKVGIiEgoJQoREQlVG3cA5TBnzhxfuHBh3GGIiKTG5s2b97p7Y77nKjJRLFy4kE2bNsUdhohIapjZS4WeU9eTiIiEUqIQEZFQShQiIhJKiUJEREKVPVGY2b1m1mVmz2Yd+0cz22VmW4KP6wp87XIze97M2s3sU+WOVUREhouiRfEtYHme419198uCjw25T5rZBODfgBXAhcCNZnZhWSMVEZFhyp4o3P0xYP8YvnQp0O7uO939JPA94PqSBiciIiOKc4zio2b2TNA1NTPP8/OAjqzHncGxvMxspZltMrNN3d3dpY5VJHU6DnZw24bbWHr3Um7bcBsdBztG/iKRPOJKFN8AWoDLgD3AV/KcY3mOFdw8w93XufsSd1/S2Jh3caFI1eg42MHiuxazdvNaNu7eyNrNa1l812IlCxmTWBKFu7/i7n3u3g/cTaabKVcn0Jz1eD6wO4r4RNJuzRNr6DnZw6n+UwCc6j9Fz8ke1jyxJubIJI1iSRRmdnbWw3cDz+Y5bSNwnpmda2aTgBuAB6OITyTt2na1DSaJAaf6T/HUrqdiikjSLIrpsfcBvwHON7NOM/sgsMbMtpnZM8DVwN8F5841sw0A7t4LfBT4GbAd+IG7/67c8YpUgmXzljGxZuKQYxNrJrJ0Xr7Gu0g4q8Q9s5csWeIqCijVbGCMYqD7aWLNROon1bN11VaaZzSP/A2k6pjZZndfku85rcwWqUDNM5rZumort15+K0vnLuXWy29VkpAxq8gy4yKVouNgB2ueWEPbrjaWzVvG6itWF32xb57RzB3X3VHmCKUaKFGIJFRu99GWP21h/bb1ahlI5NT1JJJQmuIqSaFEIZJQmuIqSaFEIZIg2WU3jvcep9aG9g4nZYqryoNUF41RiCRE7phEbU0tfd5HrdXS672DU1xXX7E6UXFq7KTyqUUhkhC5YxK9/b3U1tTy2sbXJmqKq8ZOqo9aFCIJUWhMYmrtVNpuaYspquE0dlJ91KIQSYi0lN1IS5xSOkoUIgmx+orV1E+qH7wIJ2VMIlda4pTSUaIQSYi0lN2IIk7NqkoWFQUUkURRQcN4qCigiKSGZlUljxKFiCSKZlUljxKFiCSKZlUljxKFiOQV14ByOWdVaZB8bDSYLSLDxD2gPLAPx+MvP06/91NjNVy54MpR7ceR73tqkLywWAezzexeM+sys2ezjn3ZzJ4zs2fM7H4zO6PA174Y7K29xcx05ReJSNwDys0zmll9xWpePvgyz+19jq2vbGXt5rUsvmvxmFsBcb+mNIui6+lbwPKcY48CF7v7pcAfgH8I+fqr3f2yQplORMKNpbslCQPKpb6wJ+E1pVXZE4W7Pwbszzn2iLv3Bg+fBOaXOw6RajTQ3bJ281o27t5Y9F15EgaUS31hT8JrSqskDGZ/AHi4wHMOPGJmm81sZdg3MbOVZrbJzDZ1d3eXPEiRNPrsLz/LgeMHRn1XnoQyHaW+sCfhNaVVrInCzD4D9ALrC5xyhbu/HlgBfMTM3ljoe7n7Ondf4u5LGhsbyxCtlJtmpJRWx8EOvrPtOzhDJ6wUc1eehHIipb6wJ+E1pVUks57MbCHwE3e/OOvYzcAq4Bp3P1rE9/hHoMfd/2WkczXrKX00I6X0bttwG1/f+HX66R9y3DDef+n7aZjcQNuuNpbNWzau2UTlNDD76aldT7F03tLExlkJwmY9xbIfhZktBz4JvKlQkjCzOqDG3Q8Hn18L3B5hmBKhsIHLO667I+bo0qltV9uwJAGZRPHA8w9w9NTRxO9Q1zyjWe9/AkQxPfY+4DfA+WbWaWYfBO4EGoBHg6mvdwXnzjWzDcGXngn82sy2Ak8BD7n7T8sdr8RDM1JKL18ffw01nDf7vMEkAZomKiMre4vC3W/Mc/ieAufuBq4LPt8JLC5jaJIgy+YtY8uftgxJFkmekTLQJZLkrpvVV6xm/bb1w7rzptROUVKWUUnCrCeRVM1IGeuU06gVGry9csGVmiYqo6ISHpIYaRm4vG3DbazdvHZY6+fWy29NRX96kiYO5LbMbrr0Jr7zzHcS3VKrVGGD2UoUUnU6Dnbw2V9+lofbHwaDFa0r+MLVXyj6grT07qVs3L1x+PG5S2m7pa3U4ZZFEpJybsKqtVr6vI8JNRPo7e/VzLeIJW7Wk0hcOg52cMk3LuHgiYODx7699ds88NwDbPvwtqIuSGkbT8knCbOJcme69QbFGnr7M/9q5ltyaIxCKkKxi/XWPLGGQycODTt++MThomf9pGk8JcnyzXTLpUH2ZFCLQlIvtwsjbF1A2662YSuVAfrpL/qCNDBIHHfXTdrla5nlSltLrVKpRSGpN5oqo8vmLcOwYcdrqBnVBWmg66btljbuuO4OJYkxyG2Z1VothlFbk7l/VUstOZQoJPVGs1hv9RWrmT55+rDjDZMbquKClKR6WrnTd1ctWcVvPvgbVl2+SrWYEkZdT5J6oxlcbp7RzLYPbwud9ZTkxXTjiS23i+63e37L3U/fzQVzLhj37nFjlW9Qfdn8ZZHGICPT9FhJvVKuC0jSGoNSx5Zv/ceAJL1OiUesW6GKlFspy0cnebvM8cYWNssoSa9TkkddT1IRSrUuoBzFCUvVlTXe2EaaZTTS90pyl5yUlxKFSJZSL6YbzdTdcseWWyQwV9j3KuXrkPRR15OkWqln8ZR6MV0pu7LGG1t2F93iMxczecLkoqeiJrlLTspPLQpJrXLc5ZZ6Md14uovydfWMN7bsLrrR1HvSfiHVTYlCUqtcu+KFjXeMtp/+wsYL2bx785Cd5orpLgpLgqWqezSacZ1KqG8lY6euJ0mtqO9yR7sPRcfBDh547oFh25FOmzhtxO6ipHX1VEp9qyQtOEwTJQpJrXxbfZbzLne0F+81T6zh6KmhW8IbxrvOf9eI3UVJ6+op5RTkuKRlw6kkUqKQ1Ir6Lne0F+985zvO9r3bR/xZUSfBYqS9vlXSWmlpEkmiMLN7zazLzJ7NOjbLzB41sz8G/84s8LU3B+f80cxujiJeSYeo73JHe/Eez8W+Urp6kiRprbQ0iapF8S1gec6xTwG/cPfzgF8Ej4cws1nA54FlwFLg84USilSnKO9yR3vxHs/FPqokWE199klspaVFZLWezGwh8BN3vzh4/DxwlbvvMbOzgV+5+/k5X3NjcM6tweO1wXn3hf0s1XqSchntFqJJ2HK0kCTXtSqHanu9o5XUrVDPdPc9AEGyaMpzzjwg+xanMzg2jJmtBFYCLFiwoMShimSMtlRIFFuOjrW0RrmmFyeVNpwau6Svoxi+wwx5ticD3H0dsA4yLYpyBiWSFONZdJiEPvtiklwpa0wlYa/wNIozUbxiZmdndT115TmnE7gq6/F84FcRxCajpIJx8SjUKnjbd9/GlNopoe9FoSKBx3qP0XGwo+zvXzFJTjWmkiHO6bEPAgOzmG4GfpznnJ8B15rZzGAQ+9rgmCSI5qfHp1CrYFvXthHfi4HB9oF6TwO2d2+P5P0rZrqqprQmQ1TTY+8DfgOcb2adZvZB4IvAW8zsj8BbgseY2RIz+yaAu+8HvgBsDD5uD45Jgug/c3zyzeTJVui9GGgBNs9oZvqkoVvD9npvJO9fMV1fj738WOzdYxJR15O731jgqWvynLsJ+FDW43uBe8sUmpRAEvq6q9VIpcNh+HuR251TzNeUw0j1ozoOdvD83ueHfV2t1WpKa8S0MlvGTfPT45O73uKSpkuotaH3f7nvRW4LMJ8o3r+R1pmseWINfd437Osm1EzQwsOIKVHIuBX6D3/TpTelejFXEhajFRND9qLDh/77QzRMbghd5Be2JWqhrymHkRYVtu1qo7e/d9jXXTDnAg1kRyyyBXdR0oK76OUuLLvp0ptYsX5Fahc3JWFx1lhjGGmR320bbmPt5rXDunwumHMBU2unJmZ9QaE4b738Vk1xLYOwBXdKFFIWaf9PnoT4yxVDEpJgMdISZ6UISxTqepKySPsAdxLiL1cMaSkZnpY4q0HSV2ZLSqV9R7QkxF9sDGNZ7JiWFcppibPSqetJyiLt3QZJiL+YGJIQp1QGdT1J5NLebZCE+IuJQYsdJQpqUYik2NK7l7Jx98bhx+cupe2WthgiknI4fqqPF/cdob2rZ/DjrRedxTsWz4Xp0+Hw4eFf1NAAhw4V/TOSWmZcRMapJGMpJbrQyPgdPHaK9q4ednT3sCNICDu6e3h5/1H6g3t6M2ieOY1l587KHMj33oUdHwMlCpEUyy3hMabFchFcaOQ0d6fr8IkhrYP2rh7au3voPnxi8LxJE2o4d04dF82dwTsvm0drUz2tjfUsaqxjysQJkcasRCGSYtqMJ7n6+p2X9x8dlgx2dvVw+MTpFecNk2tpaarnTa9ppKWxPpMQmuppnjmV2gnJGEbWGIVIGaRqfw7Ltz9YoAKvD6V2/FQfO7qDbqIgGezoOsILe49wsq9/8LymhsmDSWCgddDSVE9Tw2Qs7D0YSYneP41RiERIm+1UpgNHTw6OGWS3EDpfPTZ4Pa4xWDBrGq1N9Vx1wekWQktjPTOmFi4Hn3RKFCIlVm17UVcSd2fPwePDEsKO7h729pwcPG9ybQ2LGuu5rHkm73l9cyYZNNWxcHb04wc0NBSejFAiShQiJZaE8h+jEsGFJmlO9fXz0r6jQ2cYBf8eOXm6tPmMqRNpbarnmgvOHEwGrY0NzJs5lQk14+guKqUIZqYpUYiUWBLKf4xKBU+BPXqylx2vvZz2KbNon91M++xmdsyez4sz59I74fTl7+wZU2htque9S5ppCcYPWpvqmVM/aXzjBxVCg9kiJaayGhlRDujv6zkxOGawo+vIYOtg14Fjg+dM6O/jnFf30LKvg9Z9nbTu66D14R/S0lRP/WTdMyeyzLiZnQ98P+vQIuBz7v61rHOuAn4MvBAc+pG73z7S91aikLiNtCdEpStHsuzvd3YdOHZ6hlHWGMKrR0+33qZMrDk9zbSxnta/+Sta93Vwzqt7mJS7EVIF3iiPVSJnPbn788BlAGY2AdgF3J/n1Mfd/e1RxiYyXtVe9XQ8A/one/t5KbtcRZAQdnYf4dip0+MHM6dlxg+WX3wWLcFU09bGeuadMZWa7PGDP/xXWV5jNUlKe+saYIe7vxR3ICIyfsUM6Pec6B2y9mDg85f2H6Wv//Sd/rwzptLaVM8bFs3Omm5ax+z6yZG9nmqXlERxA3Bfgef+zMy2AruBT7j77/KdZGYrgZUACxYsKEuQIlKcwQH9vlPUcAYT+5uZwjlMO7aCm77ZRntXD386dHzw/IkTjIWz63jNmQ1cd8nZg4vSFjXWMW1SUi5T1Sv2d8DMJgHvBP4hz9NPA+e4e4+ZXQc8AJyX7/u4+zpgHWTGKMoUrsQsVSueq0hfv9P56unppv0HbqTpeAs1fWdTQ/3geX/aW0ND0yn+vGV2pqso+FgwaxoTy1Wuogqn/5Za7LOezOx64CPufm0R574ILHH3vWHnaTC7Mmk20TiUqEJsvnLX7V09vLD3CCd6T5ermFM/ifmzJvLqqe3sP/l7Lpo7m0+88X1cPn+RppsmVCIHs7PcSIFuJzM7C3jF3d3MlpLZaGlflMFJcmjF8ziMskLsoeOnTq9KzhpU7sgud+39zD/YReu+Dq7c10HLvk5aj+2jdct/cca0ScF3uqrkL0WiF2uiMLNpwFuAW7OOrQJw97uA9wAfNrNe4Bhwg8fdBEqTCttnIHUrnsuhhO+pA6/Uz2ZH+94hrYMd3T105Sl3ffHcGVx/2TxaGutofeuVLNq/m6m9J4Z/48EkIZUi1kTh7keB2TnH7sr6/E7gzqjjqhgVts9A6lY8l8MY3tPevn46Zs6lffb8wdXJ7bOb2Tl7Pocn18E3MzvhDZS7fuNrGgcL2RUsd931Qp6fJJUq9jGKctAYRaDCykdrjILQ9/T4yd5h5a7bu3p4ce/RoeWuD+87vTJ5XwetP/gWrU31NI6m3HWF/W1JQldml5MSRaAC/zNX+4pnzDgwpX5Iy2BH0FLonHlW3nLXLU31tK6+jZb9nbTs62TGiSNDv+dY/hYq8G+r2ilRVCv9Zx6zuKfhZpe7zl6dvOPZneyrO2PwvMmnTrBo/y5a93XQ8rcfGpxuOqzcdanHq/S3NXoJHzNUoqhW+s88JlF2ceWWu84ue51d7nr6lNpMEvjJDzLdRXsz3UbzDnUzwYNupSjf04Rf9BIp4f8fkz49VspFC43GpBzTcI+e7A2qmh4eMsPopX1H6c0qVzFiuetPrUjGe6pkUFWUKCqZ/jOPyXim4WaXux5IBju7jwwtd11jnDN7Gi2N9Vx70VmDyaCoctd6TyUGShQiOUaahjtQ7npgz4NC5a6nTpxAS1Md/23hTG5sah6cbnrO7Dom1Y5QrqISunYq4TUIoDEKSZC4B5Cz41h812J6ThyHvjlMYSHTrIV3LPoQuw/0Dyt3PatuEq2NmW0yB/dBaKpn7oycctejkfD+7KJUwmsopYT/PjRGIYmXO4C85U9bWL9tfSRrJA4fP8WO7iNDBpQvsu/TefQ4zuk7/9/vPkFLUz3Lzp09mAxam+qZVZeAlci6e0++FI8ZhiYKM5sONLr7jpzjl7r7M2WNTKpKues4uTvdwfjBju4jp+sXFSh3fdHZs7h+cX16yl1X2Cr8ipTihF3wL9/M/hL4GtBlZhOBv3L3jcHT3wJeX/7wpFqUqo5Tdrnr3O0yDx0/vQ1m3aQJtDTVj73cte7gw02fHncEUkJht0ifBi539z1B5db/MLNPu/uPANUJlpIabR2n46f6eGHvkSGJoFC565bGet6xeO6Q7qKzpk8ZX7lr3cGH0++hooQlilp33wPg7k+Z2dXAT8xsPpnCkyIls/qK1azftn7YIre/ufzvefrlV8PLXRs0z5xGS2MdV543ZzAZtDTWZ5W7TpkU92ePqBJeQ5UJSxSHzKxlYHwiaFlcRWaXuYuiCE6qg7szyRq5Z/mvueeph9jRfYRptgg/eSYrvnp659vcctetwYK0RY055SoqQSV3X0X12tQ9WDJhieKT5HQxufthM1tO/m1LRUL19vXT8eqxYXsf7Ojq4fCJgfGDC2mYXEtzVjdRZuppgXLXklHJLZCxUvdgyYQlim8Da83sK+7eC2BmZwJfAc4Hbo8gPkmhYyf7Mgmge4Ry1w2TaW2q592vP906GHW5a8nQHbKUUViiuBz4Z+C3ZvYx4BLgfwJrgPdHEJsk3KtHTg4dTA4+33XgWN5y11df0DSYDBY11jNj6sR4X8B46A4+nH4/FaVgonD3V4FVQZL4ObAbeIO7d0YVnMSvYLnrrh72HTk5eN7k2hoWNdbzugUzee/lzbQ01eUvd10pdAcfTr+fihK2juIM4EvAMmA5cB3wsJl9zN1/WaoAzOxF4DDQB/TmLiG3TB/E/wl+/lEy6zmeLtXPl4x85a4HPj+aVe56xtSJtDbVc81rm7LGEBqYN3MqE8ZarkKkWFEMUGsQfJiwrqenga8DHwnGKB4xs8uAr5vZS+5+YwnjuNrd9xZ4bgVwXvCxDPhG8K+MwZETvezsPl3uOlP6uocX9x7JW+76L5c0D9k/ebDctch4jPViPJoB6rF2f2kQfJiwRPHG3G4md98C/LmZ3VLesIa4Hvh3z1QvfNLMzjCzswfWeEh++3pODNYvau/q4Y9dh9nR1cPug6fLVQyUu25trOfaC8/MKldRRLlrkfGI4mJcpXf/5RA2RlFwLMLd7y5hDE6mteLAWndfl/P8PKAj63FncGxIojCzlcBKgAULFpQwvOQabbnrpefOGtI6KKrctYydujCkQiThtvEKd99tZk3Ao2b2nLs/lvV8vn6OYSvDgwSzDjJlxssTajxO9vbz4r4jQ9YftHf1sHNvD8dPnZ5uOqtuEi2NdSy/+KzSlbuOW5ovturCkAoRe6Jw993Bv11mdj+wFMhOFJ1Adp3p+WRmYFWc7HLX2YPJL+8/Sl/W+MG8M6bS0lTPGxYlsNx1qeliKxK7WBOFmdUBNcGK7zrgWoYv5HsQ+KiZfY/MIPbBShqf+O3Lr/KVR/4wrNx1bY2xcE4d55/ZwNsuOTs95a5FyimK9RlaAzJM3FecM4H7g1k0tcB33f2nZrYKwN3vAjaQmRrbTmZ67F/HFGtZ1NbUcPj4qbGXu5bxS3P3VlqN9WIcxfuh93wYbYUqyRbF9pHl+hkJ3/pSJFvYVqi6ZRUpl0J3x1XchSHpFHfXk0i4NPcXqwtDKoRaFJJshw5lumlyP5J+EZ4+PdP1lPuhLUIlhZQoRMpB03qlgihRiIhIKCUKEREJpUQhIiKhlChERCSUEoVIOcS9hkKzrqSEtI5CpBzinr6rWVdSQmpRiJTq7l938VKhlChESrWoT3fxxUliQk1iTAmiRCEi0UpiQk1iTAmiRCEiIqGUKEQqUdyzrqSiKFGIxK0c/eNpLaYoiaREIVIqY72LV/+4JJwSheSnWSCjV467+CT+/sf7t5HEbrEkxpQgWnAn+ekuN5mi+v2H7SM+3r+NJHZ/JTGmBImtRWFmzWb2n2a23cx+Z2Yfy3POVWZ20My2BB+fiyNWkaqjGwXJEmeLohf4e3d/2swagM1m9qi7/z7nvMfd/e0xxCciIsTYonD3Pe7+dPD5YWA7MC+ueERio35wSbhEDGab2ULgdUBbnqf/zMy2mtnDZnZRyPdYaWabzGxTd3d3mSIVKYN8g+DVSpMoEin2RGFm9cAPgY+7e+6I0tPAOe6+GLgDeKDQ93H3de6+xN2XNDY2li/gaqFZIPFK8u+/nLFpbCSRYk0UZjaRTJJY7+4/yn3e3Q+5e0/w+QZgopnNiTjM6qQFW/GK+/cflgzijk0iF+esJwPuAba7+78WOOes4DzMbCmZePdFF6VIhRqpi0fJQLLEOevpCuB9wDYz2xIc+zSwAMDd7wLeA3zYzHqBY8AN7tXcgStSIurikVGILVG4+68BG+GcO4E7o4lIRETyiX0wW6TkNHMmvZI8iF/FlCik8qhbJXqlSs4aG0kkJQoRGT8l54qmRCFSjdTFI6Og6rEi1UhdOTIKalGIhNHAuIgShVSgUnarqO9dRF1PUoHUrRK9QhsaacyjIqhFIZI2SewO07TWiqZEIZI26g6TiClRiIhIKCUKkTBabyCiRCESSn3vpZXE8RUZkRKFpEOlXWAq7fUUS+MrqaREIelQaReY8bwedYdJxLSOQiRt1O0lEVOLQoaq1i4RESlIiUKGqrQunkqiJC4xiTVRmNlyM3vezNrN7FN5np9sZt8Pnm8zs4XRRyklpYvd2FVCEtf4SirFlijMbALwb8AK4ELgRjO7MOe0DwKvunsr8FXgS9FGKSU31otdpV1gKu31FEvTjVMpzhbFUqDd3Xe6+0nge8D1OedcD3w7+Pz/AdeYmUUYoyRFpV1gKu31SEWLM1HMAzqyHncGx/Ke4+69wEFgdr5vZmYrzWyTmW3q7u4uQ7giItUpzkSRr2XgYzgnc9B9nbsvcfcljY2N4w6ualVrl4iIFBRnougEmrMezwd2FzrHzGqBGcD+SKKrVuoSSS4lcYlJnIliI3CemZ1rZpOAG4AHc855ELg5+Pw9wC/dPW+LQhJuYLZTIbrYjUxJXGIS28psd+81s48CPwMmAPe6++/M7HZgk7s/CNwD/IeZtZNpSdwQV7wyTmGzmpT7RRIt1hIe7r4B2JBz7HNZnx8H3ht1XCKJNH164e1G1aqQMtLKbJG0qIQFd5JKShQiIhJKiUJEREIpUUg0NLVTJLW0H4VEQ4OtIqmlFoVIWqhVJjFRi0IkLdQqk5ioRSEC2idDJIQShQhojYJICCUK0N2klI/+tqQCKFGA7iarXTkv2vrbkgqgRCGii7ZIKCUKEREJpUQhEkZrFESUKERCae2CiBIFoBWvUth4Zy3pb0sqgFZmg+4aq11DQ+ENgcY7a0l/W1IB1KKQdCjnegTtRS0SKpYWhZl9GXgHcBLYAfy1ux/Ic96LwGGgD+h19yVRxikJovUIIrGJq0XxKHCxu18K/AH4h5Bzr3b3y5QkRETiEUuicPdH3L03ePgkMD+OOEREZGRJGKP4APBwgecceMTMNpvZyrBvYmYrzWyTmW3q7u4ueZBSpTRrSaR8YxRm9nPgrDxPfcbdfxyc8xmgF1hf4Ntc4e67zawJeNTMnnP3x/Kd6O7rgHUAS5Ys8XG/ABHQgLYIZUwU7v7msOfN7Gbg7cA17p73wu7uu4N/u8zsfmApkDdRSIULm8IqImUVS9eTmS0HPgm8092PFjinzswaBj4HrgWejS5KSRRNYRWJTVxjFHcCDWS6k7aY2V0AZjbXzDYE55wJ/NrMtgJPAQ+5+0/jCVdEpHrFso7C3VsLHN8NXBd8vhNYHGVcUkGmTy/cVaVWiMioJGHWk0jpaYGeSMkoUYiISCglChERCaVEISIioZQoREQklBKFVCaV3hApGW1cJJVJU2BFSkYtChERCaVEISIioZQoREQklBKFiIiEUqIQEZFQVmAriFQzs27gpbjjGIU5wN64gxgDxR2dNMYMijtK4435HHdvzPdERSaKtDGzTe6+JO44RktxRyeNMYPijlI5Y1bXk4iIhFKiEBGRUEoUybAu7gDGSHFHJ40xg+KOUtli1hiFiIiEUotCRERCKVGIiEgoJYqEMLMvmNkzZrbFzB4xs7lxx1QMM/uymT0XxH6/mZ0Rd0wjMbP3mtnvzKzfzBI/BdLMlpvZ866EQy4AAAOxSURBVGbWbmafijueYpjZvWbWZWbPxh1Lscys2cz+08y2B38fH4s7pmKY2RQze8rMtgZx/++S/wyNUSSDmU1390PB538LXOjuq2IOa0Rmdi3wS3fvNbMvAbj7J2MOK5SZvRboB9YCn3D3TTGHVJCZTQD+ALwF6AQ2Aje6++9jDWwEZvZGoAf4d3e/OO54imFmZwNnu/vTZtYAbAbelYLftQF17t5jZhOBXwMfc/cnS/Uz1KJIiIEkEagDUpHB3f0Rd+8NHj4JzI8znmK4+3Z3fz7uOIq0FGh3953ufhL4HnB9zDGNyN0fA/bHHcdouPsed386+PwwsB2YF29UI/OMnuDhxOCjpNcPJYoEMbN/MrMO4H8An4s7njH4APBw3EFUmHlAR9bjTlJw8Uo7M1sIvA5oizeS4pjZBDPbAnQBj7p7SeNWooiQmf3czJ7N83E9gLt/xt2bgfXAR+ON9rSR4g7O+QzQSyb22BUTc0pYnmOpaG2mlZnVAz8EPp7T0k8sd+9z98vItOiXmllJu/u0FWqE3P3NRZ76XeAh4PNlDKdoI8VtZjcDbweu8YQMeo3id510nUBz1uP5wO6YYql4QR//D4H17v6juOMZLXc/YGa/ApYDJZtIoBZFQpjZeVkP3wk8F1cso2Fmy4FPAu9096Nxx1OBNgLnmdm5ZjYJuAF4MOaYKlIwKHwPsN3d/zXueIplZo0Dsw3NbCrwZkp8/dCsp4Qwsx8C55OZjfMSsMrdd8Ub1cjMrB2YDOwLDj2Z9NlaZvZu4A6gETgAbHH3t8YbVWFmdh3wNWACcK+7/1PMIY3IzO4DriJT+voV4PPufk+sQY3AzP4CeBzYRub/IcCn3X1DfFGNzMwuBb5N5u+jBviBu99e0p+hRCEiImHU9SQiIqGUKEREJJQShYiIhFKiEBGRUEoUIiISSolCpAyCSqQvmNms4PHM4PE5ZvZTMztgZj+JO06RYihRiJSBu3cA3wC+GBz6IrDO3V8Cvgy8L67YREZLiUKkfL4KvMHMPg78BfAVAHf/BXA4zsBERkO1nkTKxN1Pmdn/An4KXBuUCRdJHbUoRMprBbAHSMXmPSL5KFGIlImZXUZmZ7o3AH8X7KAmkjpKFCJlEFQi/QaZPQ1eJjOA/S/xRiUyNkoUIuVxC/Cyuz8aPP46cIGZvcnMHgf+L3CNmXWaWWIr14qAqseKiMgI1KIQEZFQShQiIhJKiUJEREIpUYiISCglChERCaVEISIioZQoREQk1P8HPSkp4q8NHwwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plotBestFit([4,0.4,-0.6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def stocGradAscent0(dataMatrix, classLabels):\n",
    "    m, n = np.shape(dataMatrix)\n",
    "    alpha = 0.01\n",
    "    weights = np.ones(n)\n",
    "    allweights=[]\n",
    "    for i in range(m):\n",
    "        h = sigmoid(sum(dataMatrix[i] * weights))\n",
    "        error = classLabels[i] - h\n",
    "        weights = weights + alpha * error * dataMatrix[i]\n",
    "        allweights.append(weights)\n",
    "    return weights#,allweights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "weights=stocGradAscent0(np.array(dataMat),labelMat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXyU9bX48c8JCWELYd9CAgghSBEQYlApIiprrVbrWrVae0W87lXpbfvrctvb160Bd1TApdSKW1u1XqusioAim4KoEAhrCDtICASSTHJ+f2SwIcxMtpl5nmfmvF+vvMg880zmJDN8zzzf811EVTHGGGOCSXA6AGOMMe5micIYY0xIliiMMcaEZInCGGNMSJYojDHGhJTodACR0KFDB+3Zs6fTYRhjjGesXr36gKp2DHRfTCaKnj17smrVKqfDMMYYzxCR7cHus64nY4wxIVmiMMYYE5IlCmOMMSFZojDGGBNSxBOFiLwoIvtE5Mtqx34nIoUissb/NSHIY8eJSJ6I5IvIf0U6VmOMMaeLxhXFLGBcgOOPqepg/9d7Ne8UkSbA08B4oD9wvYj0j2ikxhhjThPxRKGqi4FDDXhoDpCvqltUtQx4Dbg8rMEZY4yplZM1irtE5At/11TbAPenAQXVbu/0HwtIRCaKyCoRWbV///5wx2qM5xQUFXD3e3eT81wOd793NwVFBbU/yJgAnEoUzwK9gcHAbuCRAOdIgGNBN89Q1Zmqmq2q2R07BpxcaEzcKCgqYND0QcxYPYOVu1YyY/UMBk0fZMnCNIgjiUJV96pqhapWAs9R1c1U004gvdrt7sCuaMRnjNflfpzL0bKjlFeWA1BeWc7RsqPkfpzrcGTGixxJFCLStdrNK4AvA5y2EsgUkV4i0hS4DngnGvEZ43XLC5d/myROKq8sZ0XhCociMl4WjeGxrwLLgCwR2SkiPwVyRWSdiHwBjALu95/bTUTeA1BVH3AXMBdYD7yhql9FOl5jYsGwtGEkJSSdciwpIYmctEAX78aEJrG4Z3Z2drbaooAmnp2sUZzsfkpKSKJV01asnbSW9NT02n+AiTsislpVswPdZzOzjYlB6anprJ20ltuH3k5OtxxuH3q7JQnTYDG5zLgxsaKgqIDcj3NZXricYWnDmDx8cp0b+/TUdJ6a8FSEIzTxwBKFMS5Vs/tozZ41zF43264MTNRZ15MxLmVDXI1b2BWFMS5Svatp++Htrh3i2pguMeM9liiMcYmaXU0JAS743TDE1brE4o91PRnjEjW7miqpBED8q9mcHOI6efhkx2IE6xKLR3ZFYYxLBJpNDdCxZUd6pvYkJy3HFV08Nus7/liiMMYlhqUNY82eNac0wkkJSVzT/xpXDXMNFqfTXWImcqzryRiXmDx8Mq2atvp26Q23dDXV5JU4TfhYojDGJbwymzoacdpeGu5iaz0ZY1zF1qlyhq31ZIzxDBtV5T6WKIwxrmKjqtzHEoUxxlVsLw33sURhjAnIqYJyJEdVWZG8YayYbYw5jdMF5ZNrSS3ZsYRKrSRBEhiRMaJREw6d/p3cztFitoi8KCL7ROTLasemiMgGEflCRN4SkTZBHrvNv2XqGhGxlt+YKHG6oJyems7k4ZPZUbSDDQc2sHbvWmasnsGg6YMafBXg9O/kZdHoepoFjKtxbD4wQFUHAhuBX4R4/ChVHRws0xljQmtId4sbCsrhbtjd8Dt5VcQThaouBg7VODZPVX3+m58C3SMdhzHxaPnO5WQ+lcm0ldNYuWsl01dNr9OncjcUlMPdsLvhd/IqNxSzbwXeD3KfAvNEZLWITAz1Q0RkooisEpFV+/fvD3uQxnhNQVEBI2eNpLSi9NtjPvVRXFpc66dyNyzTEe6G3Q2/k1c5mihE5FeAD5gd5JThqjoEGA/cKSIXBPtZqjpTVbNVNbtjx44RiNZEmo1ICa/cj3NPSRIn+dRX66dyNywnEu6G3Q2/k1dFZdSTiPQE3lXVAdWO3QxMAi5W1ZI6/IzfAUdVdWpt59qoJ++xESnhl/NcDit3rQx4380DbyYlOcX1O9SdHP20onCFa5ZZj1WhRj05ssy4iIwDfg6MDJYkRKQlkKCqxf7vxwC/j2KYJopCFS7dtMS2lwxLG8bnez7HV+k75XjThKa8nfc2JeUlrt+hLj013V5/F4jG8NhXgWVAlojsFJGfAtOAFGC+f+jrdP+53UTkPf9DOwNLRWQtsAL4l6rOiXS8xhleG5HihW6yycMnk9I0hcSEf38eTG6SzGVZl32bJMCGiZraRfyKQlWvD3D4hSDn7gIm+L/fAgyKYGjGRby0GY5X9ow+2Sdfs+vmh2/80FNJ2TjPdrgzrjB5+GRmr5t9Wo3CjSNSvNRNFqjrxk1J+WQN4mSt5MaBN/LyFy+7vnYSb2wJD+Ma0SpcFhQV8OsPfs37+e+DwPg+4/nDqD/U+bmCFYlzuuWw/Lbl4Q437NwycKBmHImSSIVW0CShCb5Knw1oiDLXFbONCSQahcuCogLOevYsikqLvj32l7V/4e0Nb7PujnV1apDc9Im8IYJ1SUW7Ma55Zebzz8E9WXx385VavHHDhDtjGq2uxeXcj3M5UnrktON1mYR2UixM3DqZlJfftpynJjzlyCf2QAMYarLaiTvYFYXxvPoUl5cXLkc5vbu1kso6N0hu+UTudYGuzGry0pVaLLMrCuN59Vk8bljaMAQ57XgCCfVqkNzwidzral6ZJUoignw7nNeLV2qxyhKF8bz6zMGYPHwyrZNbn3Y8JTnl2wbJzXMkGhubm363mktqTMqexLKfLmPS0Em2xIbL2Kgn43l3v3c3M1bPOK24fPvQ2wMWQUONenLLiKBAGhtboFFGTRKa0K9Dv0ZvCmS8z9GNi4yJtPoWl9NT05l1xSz2PrSXvQ/uZdYPZn3bQLp5c5vGxhZolFFpRWlYNgUysc0ShfG8cK4KGomlRMLV3dPY2EKNMnJTQjTuY6OeTEwI1xyMcM+RCOdyH42NrbZRRrUlnZqzqK2rKn7YFYXxtHAXZ8M9RyKcXVmNja3m42sKlXROJrwZq2ewctdK66qKM5YojGdFovEK9+Y2jekuqpkEgUbFVv13G9R5EMlNkus8FNXNtRsTedb1ZDwrUovzherGqm/3S/+O/Vm9azWVVH57rC7dRaG6rML1u9VnbS2vLQNvwssShfGsaDde9a03FBQV8PaGt09JEgAtklrU2l0UjRVq61PX8fr6VidZnaVhrOvJeNawtGGn9bdHsvGqb/dL7se5lJSfuoGjIPwg6we1Nk5u+wQfC+tbWZ2l4SxRGM+KduNV38Y70PmKsv7A+lqfK9pJsDbhrt04weosDReVRCEiL4rIPhH5stqxdiIyX0Q2+f9tG+SxN/vP2SQiN0cjXuMN0W686tt4N6axd+MneK+vb+W2qzQvidYVxSxgXI1j/wUsVNVMYKH/9ilEpB3wW2AYkAP8NlhCMfEpmo1XfRvvxjT20UqCblr7KdLcdpXmJVFb60lEegLvquoA/+084EJV3S0iXYFFqppV4zHX+8+53X97hv+8V0M9l631ZCKlvrvwRWPXvoYWaN28rlUkxNvvW1+h1npyMlEcVtU21e7/RlXb1njMg0AzVf0f/+1fA8dVdWqAnz8RmAiQkZExdPv27ZH6VYxxjcY0fvVdTDEWRGu7XS/y8laop28cQIBdZwBVnQnMhKorikgGZU5nww6dEaxA+71XvkezxGYhX4tgffZvfP1G1F6/urxvwvneisZ2u7HIup5Mo9klvXNynsth5a6VQe8P9VoEuqKAqiG8bZq1ifjrV5f3jb23glNVRAJ9lm4Yty4z/g5wchTTzcA/A5wzFxgjIm39Rewx/mPGRWzYoXMCFWirC/VanCy2J9RoBhSNyutXl/eNvbdOt25nETe9sJx3v9gdteeM1vDYV4FlQJaI7BSRnwJ/AkaLyCZgtP82IpItIs8DqOoh4A/ASv/X7/3HjIvYsEPn1LbQHwR+LU5256SnptM0sWmdHhNudXnfLN6x2N5bflv2H+XOVz7j+9OW8mVhEWW+ytofFCZRqVGo6vVB7ro4wLmrgP+odvtF4MUIhWbCIFaWd/Cik8NoTxZoj/uOs37/enzq+/acmq9Fze6cmlcUgR4TCbW9bwqKCsg7kHfa4xIlMa7eW3uKTvDEwk28saqA5MQE7rmoD/9xwRm0bhb8w0G42VaoptGC9SO/f8P7vPzFy54tcLuhQF/fGOrSpx+qNqFo1OoAtcV693t3M331dHyVvlMel9wkmU13b/LUe6khDpeU8eyizcz6ZBuVqtwwrAd3jupDx5TkiDyfK4bHRpMliuirOezwxoE3Mn72eM8WId1QRG1oDLUNAQ1WAO/UshM9U3tGddhoqFiDxTmo8yDWTFoT8dicUlLm488fb2P6R5s5WurjirPTuP+SvqS3axHR5/Xy8FjjETWHHd793t0RX/00kqKxemukYqhtCGiwLp9r+l8T9dcmVKzB4hyRMSJa4UVVma+S11fu4ImF+Rw4WsolZ3bmobFZZHVJcTo0WxTQRIbXC9xuiD9SMbhxHalAvBJnY1VWKm9/Xsglj37Er//5FWd0aMk/7jiP52/OdkWSAEsUJkK8vq6OG+Kvawz1Xa/JKyvBeiXOhlJVPtiwlwlPLuG+19fQMjmRP//kHF6//VyG9mjndHinsBqFiQg39PE3hhvitwlpsWvltkPkztnAym3fkNGuBQ+M6cv3B3YjISF8E+jqy4rZxhFeX1fHDfHXFkM8rtfkZRv2HGHKnDwWbthHx5Rk7rk4k2uz02ma6HznjiUKY2JUsJFBOd1yWH7bcgciMoHsOFjCYws28vaaQlolJ3LHhb255fyetGgahvFErVtDcfHpx1NS4MiROv8YG/VkTIwKy2THMDU05nT7i0uZ9sEmXlmxgwQRJl5wBneM7E2bFqfPhm+wQK9dqOMNYInCGA+bPHwys9fNPq1GUa+RQVFoaOLNkRPlzPxoCy8s3UpZRSXXnpPOPRdl0iW1mdOhNYglCmM8rOYSHl6sBcWSE+UVvLRsG88s2szhknIuHdiVB8Zk0atDS6dDaxRLFMZEQDSX/7A9Fpznq6jk76t38viCTew5coKRfTvy0NgsBqSlOh1aWFiiMCbMag5ZXbNnDbPXzbYhqzFIVXn/yz1MnZvHlgPHODujDY9dO5jzerd3OrSwskRhTJi5YfkPE1mqytL8A+TOyWNdYRGZnVox46ahjOnfOaybCdVJSkrwwQhhYonCmDBzw/If9RKFhsZRYR7VtabgMLlzNvDJ5oOktWnO1KsHccXZaTRxarJcFEamWaIwJsw8tz9HrA+BDdOorvx9xUydu5E5X+2hfcum/ObS/txwbgbJiU3CEKS7OTYdUESyRGRNta8jInJfjXMuFJGiauf8xql4jamreFnMrjb1XYPKrQoPH+ehv61lzGOLWZp/gPsv6ctHk0dx63d7xUWSAJfMzBaRJkAhMExVt1c7fiHwoKpeWp+fZzOzjdPcsPyHk1y1BlWomkGI9u/QsTKe/jCfvy7bDgI/PrcH/zmqD+1ahnGynIt4YWb2xcDm6knCGC+L9yGrXi7oHy318cKSrTy3ZAslZT6uGtqdey/pS1qb5k6H5hi3JIrrgFeD3HeeiKwFdlF1dfFV9MIyxjSE5wr6QKmvgleW72DaB/kcPFbGuO904cGxfenTKUaK+o3geKIQkabAZcAvAtz9GdBDVY+KyATgbSAzyM+ZCEwEyMjIiFC0xmlu2Mfa1M5VBf1aRnVVVCpvfV7IY/M3Unj4OOed0Z6fj+/H4PQ2UQ7UvRyvUYjI5cCdqjqmDuduA7JV9UCo86xGEZtc1e/tNVFe+M8Lr5WqMv/rvUyZm8emfUc5Ky2VyeOy+G6fDtGfC+ECbq9RXE+QbicR6QLsVVUVkRyqRmkdjGZwxj283O/tuCgv/Of2NaiWbT5I7twNfL7jML06tOTpHw1h/IAujm4c5GaOJgoRaQGMBm6vdmwSgKpOB64C7hARH3AcuE6dvgTykhhbPtqL/d5h55bXtA5xuLGg/2VhEblz81i8cT9dWjfjT1eexVVDu5PYxPmNg9zM0UShqiVA+xrHplf7fhowLdpxxYwYWz7aVf3eTnHLa+qWOOpo64FjPDIvj3e/2E1q8yR+OaEfPz6vJ82S4mMeRGO5oevJmDoJy94LJq7sPXKCJxZu4vWVBTRtksCdo3oz8YLepDZPcjo0T7FEYTwjmv3eNrrK24pKynn2o83M+mQrFZXKDcMyuOuiPnRK8ebGQU5zfNRTJNioJ78GzkiNd64esdPQ1zTctQ2XvreOl1Xw50+2Mn3RZopLfVw+qBs/G51FRvsWjsX0LbfUl4Jw+6gnY1wlJkdXuaAhiqTyikpeX1nAkws3sa+4lIv7deLBsVmc2bW106H9m8fqOtVZoohlsb58dIS4enSVW15Tl8RRWan83xe7eHT+RrYfLOGcnm155oYhZPdsF9U4Yp0lilgW458iI8UVo6tCdVO4odvQ4feWqrJo435y5+SxfvcR+nVJ4cVbshmV1SkuJ8tFmiUK4xpuKSC7YnSVh7spvhWhPvnV2w/x8Jw8Vmw9REa7Fjxx3WC+P7CbTZaLICtmG1dwWwHZ8WXCXVosrpcw/w55e4qZMjePBev30qFVMvdc3IfrzsmgaaJHJsu5/DW1YrZxPbcVkN04qzgkl4+oaYyCQyU8tmAjb31eSKvkRB4am8VPhvekRVOPNV8uqes0hMf+0iZWubqA7AWx0FVVw/7iUp7+MJ/Zy7eTIMLEC87gjpG9adPCoxsHeThhW6IwruCKAnJ9xPAn+LBo3fBhqUdOlPP84i08v3Qrpb5KrslO596LM+mSapPlnGKJwriCKwrI9RHpT/Ae7qYAGvR3OFFewV+XbeeZRfl8U1LOpQO78rPRfTmjY6sIBGjqwxKFcQW3L0sddbF8VVIj2fkqKvnHZzt5fMEmdhedYERmByaP7cdZ3VMdCtDUZInCuIbnCsimYfxJUFWZ8+UepszLY8v+YwxOb8Mj1wzi/N4dwvM81j0YNiEThYi0Bjqq6uYaxweq6hcRjcwYU3ce66r6OP8AD8/ZwBc7i+jTqRUzbhrKmP6dwztZLgYL/E4JmihE5BrgcWCfiCQBt6jqSv/ds4AhkQ/PGFMnHvmEvLZLJlOeX87S/AOktWnOlKsGcuWQ7jSxyXKuFuqK4pfAUFXd7d+G9K8i8ktVfROwV9XEN499go+6Gn+f/HbdeeSCm3g/azjtdh/h15f254ZhGbZxkEeEShSJqrobQFVXiMgo4F0R6Q6EbRqhiGwDioEKwFdzZqBUXYs+AUwASqi6svksXM9vTIN45BO8Y/x/n12Hj/PEgk38bXUBzZOacM+IM7htRC9SmjVw46Bo1B2stnGaUIniiIj0Plmf8F9ZXAi8DXwnzHGMUtUDQe4bD2T6v4YBz/r/Nca41DfHynhmUT5/WbYdFG45vxd3jupN+1bJVSc0tDGORt3BahunCZUofk6NLiZVLRaRccAvIhrVqS4HXtKqRak+FZE2ItL15NWOMcY9jpX6eHHpVmYu3sKxMh9XnN2d+0dn0r1tjY2DotEYW/dg2IRKFH8BZojII6rqAxCRzsAjQBbw+zDFoMA8EVFghqrOrHF/GlBQ7fZO/7FTEoWITAQmAmRkZIQpNGMaIY66MMp8lby6YgdPfbCJA0fLGNO/Mw+OzaJvZwcb5Rj7GzspVKIYCvwv8LmI3AucBfwMyAV+HMYYhqvqLhHpBMwXkQ2qurja/YEK56fVSPwJZiZUrR4bxviMk7zc2MZBF0ZFpfLPNYU8On8jO785zrlntGPmj/sxJKOt06GZMAqaKFT1G2CSP0ksAHYB56rqznAGoKq7/P/uE5G3gBygeqLYCVSfntvdH4uJB3HQ2HqRqrJw/T6mzM0jb28x3+nWmj9ecRYXZHawjYNiUKh5FG2Ah6kqHI+jatTR+yJyr6p+EI4nF5GWQIK/9tESGMPpXVrvAHeJyGv+WIqsPmGMc5ZvOUju3DxWb/+GXh1aMu1HZzNhQNfobBwUjbqD1TZOE6rr6TPgGeBOf41inogMBp4Rke2qen0Ynr8z8Jb/E0gi8IqqzhGRSQCqOh14j6oklU/V8NifhOF5jfk3L3dvRdFXu4qYMjePRXn76dw6mT9eMYBrstNJatKAjYMa2hhH4/Ww1/w0QXe4E5HuwbqZROQ2VX0uopE1gu1wF0OisStYpJ7D5Tua1dX2g8d4ZN5G3lm7i9TmSfznhb25+fyeNlkuxjRoh7tQtQg3JwljXMPjXRj7jpzgyQ828dqKAhKbCHeO6s3EC3qT2ryBk+WMZ9nqscbdvNzYerQLo+h4OTM+2syLH2/FV6Fcn5PB3Rf1oVNr2zgoXlmiMO7m0cbWi3WP42UVzPpkG9M/2syRE+VcPqgb94/uS4/2LZ0OzTjMEoUxkeChYb3lFZW8saqAJxZsYl9xKaOyOvLQ2H7079bw7UxNbLFEYUycqqxU3l23m0fn5bHtYAnZPdoy7UdDyOnVzunQjMtYojAmzqgqH23cT+6cPL7efYR+XVJ44eZsLurXySbLmYAsURgTR1Zv/4bcORtYvvUQ6e2a8/i1g/n+oG62cZAJyRKFMbGoRjF9Y4cMpoy4ifl9z6NDq6b892Xf4fqcDJomNmCynIk7liiMiQSnh/X6n3tn64489t0beHPARbQqO84Di//KrR++TMtk+69v6s7eLcaEq1F30ZDYAy1Sefq8a5g9eAKg3LbiLe749O+0PVEMya9FNRbjfZYojAlXI+6CIbHFJ8p5fslWnp/4HMeTkrl63QLu/fhVuhUH20DSAS5KqN9yY0wuYonCmBhworyC2ct38PSH+Rw6VsaErZ/xsyUv0+dQWHcFCA8XJNQ6P7cL5704wRKFMR7mq6jkzc8LeXz+RnYVnWBEZgceGpvFwPRLnQ7NxBBLFMY4rQHdHqrK3K/2MnVeHvn7jjKoeypTrh7E8D4d/v1Yr66RZVzHEoUxTqtnt8cnmw/w8Jw81hYcpnfHljx7wxDGDehy6mQ561c3YWSJwgRmxb36i8Sn+GqN/7rOvcm96FaWZAyiW2ozcn84kCuHpJHYkI2DGsPeG3HHEoUJzIp79RehRnJzuzQeHXEj/+o3grYlRfy/753Jjef2cG7joMa+N9zYLebGmFzEsUQhIunAS0AXoBKYqapP1DjnQuCfwFb/oTdVteae2sbEpN0p7Xny/Ot5Y+Bokn1l3PPxK9y24i1SniyJ/JOHumpoLDdedbgxJhdx8orCBzygqp+JSAqwWkTmq+rXNc5boqo2hMPEjW+apfDsuVcxa+j3URFu+uxf3LXsdTqUFEUvCLuiNNU4lihUdTew2/99sYisB9KAmonCmNjm7/YoSUrmxezLmZFzJUeTW3Dllx9w39JXSD+yz+kITZxzRY1CRHoCZwPLA9x9noisBXYBD6rqV0F+xkRgIkBGRkZkAjUmAsoOHea1lTt4cmE+B46WMrp/Zx584CqyDmx3OjRjABckChFpBfwDuE9Va3YUfgb0UNWjIjIBeBvIDPRzVHUmMBMgOztbIxhyfLDiXsRVVirvrN3FI/PzKDh0nJxe7Zhx01CG9mgLdx0K/CA3/P0j+d6wEVWu5GiiEJEkqpLEbFV9s+b91ROHqr4nIs+ISAdVddHCNTHK/lNGjKryYd4+cufksWFPMf27tmbWTwYwsm/Hf8+FcPPfP5KxWW3ElZwc9STAC8B6VX00yDldgL2qqiKSAyQAB6MYpjFhtXLbIXLnbGDltm/o2b4FT11/Nt87qysJ0d44qLZP7nZFaapx8opiOHATsE5E1viP/RLIAFDV6cBVwB0i4gOOA9epqnUrGc9Zv/sIU+fmsXDDPjqlJPM/PxjAteekkxTtyXIn1fbJ3c1XNCbqnBz1tBQI+TFKVacB06ITkTHht+NgCY/Oz+Ofa3eRkpzIz8f145bze9K8qUOT5YxpAMeL2caEnQsKovuKTzDtg3xeWb6DxCbC7Rf05o6RvUltkRSV5zcmnCxRmNjjYEH0yIlyZn60hReWbqW8opLrctK556JMOrVuFvHndlS4krPVRlzJEoUxYXCivIKXlm3jmUWbOVxSzmWDuvGz0X3p2aGl06FFR7iSs9VGXMkShTGN4Kuo5G+rd/LEgk3sOXKCC7M68uCYLAakpTodWmj2yd3UgyUKY0IJ0qVSmdKa95du4JF5eWw5cIwhGW14/LrBnHtGeweCbAD75G7qwRKFMaHUSBIKLOl5NlMu+DHrXvmMvp1b8dyPs7nkzE6nbhxkTAyxRGFiT4S6VT7v2pfckTezrMcguh/ewyNXD+IHZ6fRJNqT5YyJMksUJvaEuVtlU/t0po64iblZ59P+2GF+N38616+dQ/L08rA+T525YPhvwOe2mkfMskRhTBCFh4/z+Ph7+ceAi2hRXsrPlrzMrav+Sauy484G5sb1kKzmEdMsURhTw8GjpTyzaDN/XbYd+o/k1lXv8J+f/o12x60xNPHJEoUxfkdLfTy/ZAvPL9lKSZmPq4emc+89l9Ftd4B9IaxLxcQRSxQm7pX6Kpj96Q6mfZjPoWNljB/QhQfG9KVPpxS4apvT4cUWN9ZXTK0sURhviEADU1GpvPnZTh5fsInCw8cZ3qc9D43tx+D0No0Mtg7itcF0Y33F1MoShfGGMDYwqsq8r/cydW4em/YdZWD3VB7+4UC+m9mhkUHWQ2N+HxthZKLMEoWJK8s2H+ThORtYU3CYMzq05JkbhjB+QBdvTZaL5SsO40qWKMypYrRL5MvCInLn5rF44366tG7Gwz88ix8O6U6iUxsHGeMhlijMqWKsD3nrgWM8Mi+Pd7/YTZsWSfxqwpncdF4PmiV5cOOgGE3ixv0cTRQiMg54AmgCPK+qf6pxfzLwEjCUqr2yr1XVbdGO04RRlBq7PUUneGLhJt5YVUDTJgncfVEfbrvgDFo38/DGQbGQxK2+4kmOJQoRaQI8DYwGdgIrReQdVf262mk/Bb5R1T4ich3wMHBt9KM1YdPQxq6ODczhkjKe/Wgzsz7eRqUqNwzL4O6LMumYktzAgJhxT9kAAA28SURBVCMkXhtMu/LxJCevKHKAfFXdAiAirwGXA9UTxeXA7/zf/x2YJiKiqhrNQI0L1NLAHC+r4M+fbGX6os0Ul/q4YnAa94/uS3q7FlEKsJ6swTQe4mSiSAMKqt3eCQwLdo6q+kSkCGgPHKj5w0RkIjARICMjIxLxGhcqr6jktZUFPLlwE/uLS7nkzE48ODaLfl1aOx2aMTHDyUQRaDxizSuFupxTdVB1JjATIDs72644GsojXSKVlcr/fbGLR+dvZPvBEs7p2ZZnbxhCds92TodmTMxxMlHsBNKr3e4O7Apyzk4RSQRSgUPRCS9OubxLRFVZlLef3Ll5rN99hH5dUvjzLedwYVZHb82FaAiPJHETe5xMFCuBTBHpBRQC1wE/qnHOO8DNwDLgKuADq094VLDRTifVobFbte0QuXPyWLHtEBntWvDEdYP5/sBuJMTLxkEuT+ImdjmWKPw1h7uAuVQNj31RVb8Skd8Dq1T1HeAF4K8ikk/VlcR1TsVrGilUkqgl92/Yc4Spc/NYsH4fHVOS+cMPBnBtdjpNE+NsspzNozAOkVj8gJ6dna2rVq1yOgxTXahuoSDvwR0HS3hswUbeXlNIq+REJo3szU+G96RF0zidJ9qAv6ExdSUiq1U1O9B9cfo/zrjZ/uJSpn2wiVdW7CBBhNsv6M2kkWfQpkVTp0MzJi5ZojCuceREOc8t3sILS7dS6qvkmux07r04ky6pzZwOzZi4ZonCOO5EeQUvLdvGM4s2c7iknEsHduWBMVn06tDS6dCMMViiMNESYGinTxL4+zmX8viURew5coIL+nZk8tgsBqSlOhSkMSYQSxQmOqqNylFV3v9yD1Pn5bFl/zHObtOMx64dzHm92zsYoAfYPArjEEsUJqqWbjrAw3M2sK6wiMxOrZh501BG9+8c+5PlwsGGwBqHWKIwUbG24DC5czfwcf5B0to0Z+rVg7ji7DSauGWynM1RMCYoSxQmovL3FTN17kbmfLWH9i2b8ptL+3PDuRkkJ7ps46BY2OvBmAixRAH2aTICCg8f54kFG/n76p00T2rC/Zf05acjetEqOc7ecvbeMjEgzv7XBmGfJsPm0LEynvkwn5c+3Q4Kt5zfiztH9aZ9K5dtHFRd6wguSW7vLRMDLFGYsDhW6uOFpVuZuXgLJWU+fjikO/eN7ktam+ZOh1Y7a7SNCckShWmUUl8Fry7fwVMf5HPwWBljv9OZh8Zm0aeTDdk0JlZYojANUlGpvP15IY/O30jh4eOce0Y7nh/Xj7Mz2jodWnjZHAVjLFGY+lFVFqzfx5S5G9i49ygD0lrzv1eexYjMDrE7FyLQ72XFaBNHLFGAzXito+VbDvLwnA18tuMwvTq05OkfDWH8gC6xvXFQY4vR9t4yMcASBdgnw1p8tauIKXPzWJS3n86tk/nfK8/iqqHdSWoSIxsHhWrMG1votveWiQGWKExQ2w4c45H5G/m/tbtIbZ7EL8b34+bze9IsyYHJcpGcjxDq8bHanWZMPTiSKERkCvB9oAzYDPxEVQ8HOG8bUAxUAL5guy+Z8Np75ARPLtzE6ysLSGqSwF2j+nDbBWeQ2jzJuaBsPoIxjnHqimI+8Av/vtkPA78Afh7k3FGqeiB6ocWvopJypi/ezJ8/3oqvQvnRsAzuuqgPnVJs4yBj4pkjiUJV51W7+SlwlRNxmCrHyyqY9ck2nl2UT3Gpj8sHdeNno7PIaN/C6dCcZ8VoY1xRo7gVeD3IfQrMExEFZqjqzGA/REQmAhMBMjIywh5kLCqvqOT1lQU8uXAT+4pLubhfJx4cm8WZXSO4pIXXWDHamMglChFZAHQJcNevVPWf/nN+BfiA2UF+zHBV3SUinYD5IrJBVRcHOtGfRGYCZGdna6N/gRhWWam8u243j87LY9vBErJ7tOXpG4ZwTs92TodmjHGhiCUKVb0k1P0icjNwKXCxqgZs2FV1l//ffSLyFpADBEwUpnaqykcb95M7J4+vdx+hX5cUXrwlm1FZndw/Wc66gIxxjFOjnsZRVbweqaolQc5pCSSoarH/+zHA76MYZkxZvf0QD8/JY8XWQ6S3a87j1w7mskHdvDNZzrqAjHGMUzWKaUAyVd1JAJ+q6iQR6QY8r6oTgM7AW/77E4FXVHWOQ/F6Vt6eYqbMzWPB+r10aJXM7y//Dtedk0HTxBiZLBeM7QNhTNg4NeqpT5Dju4AJ/u+3AIOiGVcsKThUwmMLNvLW54W0aprIQ2Oz+MnwnrRo6obxC1Fg8y6MCZs4aTXix/7iUp7+MJ/Zy7eTIMJtI87gjpG9aduyqdOhGWM8yhJFjCg+Uc5zi7fw/NKtlPoquSa7O/dcnEnXVA9sHGSMcTVLFB53oryClz/dztMf5vNNSTkTzurCA2Oy6N2xldOhGWNihCUKj/JVVPLmZ4U8vmAju4pOMCKzAw+NzWJg9zZOh2aMiTGWKDxGVZn71R6mzM1j8/5jDEpvw9SrB3F+nw5Oh+YuNu/CmLCxROEhn+Qf4OG5eawtOEyfTq2YfuMQxn6ni/snyznBhsAaEzaWKDzgi52HmTI3jyWbDtAttRm5Vw3kyrPTSIyVjYOMMa5micLFNu8/yiPz8nhv3R7atkji/33vTG48t4czGwcZY+KWJQoX2l10nCcWbOJvq3fSLDGBey/O5D9G9CKlmYMbBxlj4pYlChf55lgZzyzK5y/LtoPCj8/rwZ2j+tChVbLToRlj4pglChc4VurjxaVbmbl4C8fKfFw5pDv3XZJJ97a2cZAxxnmWKBz2Sf4B7nntcw4cLWNM/848ODaLvp1tCKcxxj0sUTisV8eW9O+Wyn2XZDIko63T4RhjzGksUTisa2pzXro1x+kwjDEmKBuIb4wxJiRLFMYYY0JyJFGIyO9EpFBE1vi/JgQ5b5yI5IlIvoj8V7TjNMYY42yN4jFVnRrsThFpAjwNjAZ2AitF5B1V/TpaARpjjHF311MOkK+qW1S1DHgNuNzhmIwxJu44mSjuEpEvRORFEQk0LjQNKKh2e6f/WEAiMlFEVonIqv3794c7VmOMiVsRSxQiskBEvgzwdTnwLNAbGAzsBh4J9CMCHNNgz6eqM1U1W1WzO3bsGJbfwRhjTARrFKp6SV3OE5HngHcD3LUTSK92uzuwKwyhGWOMqQdHitki0lVVd/tvXgF8GeC0lUCmiPQCCoHrgB/V5eevXr36gIhsD0uw0dEBOOB0EA1gcUePF2MGizuaGhtzj2B3ODXqKVdEBlPVlbQNuB1ARLoBz6vqBFX1ichdwFygCfCiqn5Vlx+uqp7qexKRVaqa7XQc9WVxR48XYwaLO5oiGbMjiUJVbwpyfBcwodrt94D3ohWXMcaY07l5eKwxxhgXsEThDjOdDqCBLO7o8WLMYHFHU8RiFtWgI06NMcYYu6IwxhgTmiUKY4wxIVmicAkR+YN/SZM1IjLPP1TY9URkiohs8Mf+loi0cTqm2ojI1SLylYhUiojrh0B6cRVl/9I8+0Qk0BwpVxKRdBH5UETW+98f9zodU12ISDMRWSEia/1x/3fYn8NqFO4gIq1V9Yj/+3uA/qo6yeGwaiUiY4AP/PNeHgZQ1Z87HFZIInImUAnMAB5U1VUOhxSUfxXljVRbRRm43u2rKIvIBcBR4CVVHeB0PHUhIl2Brqr6mYikAKuBH3jgby1AS1U9KiJJwFLgXlX9NFzPYVcULnEySfi1JMS6Vm6iqvNU1ee/+SlVS624mqquV9U8p+OoI0+uoqyqi4FDTsdRH6q6W1U/839fDKwnxEKkbqFVjvpvJvm/wtp+WKJwERH5o4gUADcAv3E6nga4FXjf6SBiTL1WUTbhISI9gbOB5c5GUjci0kRE1gD7gPmqGta4LVFEUS0r6qKqv1LVdGA2cJez0f5bbXH7z/kV4KMqdsfVJWaPqNcqyqbxRKQV8A/gvhpX+q6lqhWqOpiqK/ocEQlrd5+TO9zFnbquqAu8AvwL+G0Ew6mz2uIWkZuBS4GL1SVFr3r8rd3OVlGOIn8f/z+A2ar6ptPx1JeqHhaRRcA4Ai+22iB2ReESIpJZ7eZlwAanYqkPERkH/By4TFVLnI4nBn27irKINKVqFeV3HI4pJvmLwi8A61X1UafjqSsR6XhytKGINAcuIczth416cgkR+QeQRdVonO3AJFUtdDaq2olIPpAMHPQf+tTto7VE5ArgKaAjcBhYo6pjnY0qOBGZADzOv1dR/qPDIdVKRF4FLqRq6eu9wG9V9QVHg6qFiHwXWAKso+r/IcAv/YuTupaIDAT+QtX7IwF4Q1V/H9bnsERhjDEmFOt6MsYYE5IlCmOMMSFZojDGGBOSJQpjjDEhWaIwxhgTkiUKYyLAvxLpVhFp57/d1n+7h4jMEZHDIvKu03EaUxeWKIyJAFUtAJ4F/uQ/9CdgpqpuB6YANzkVmzH1ZYnCmMh5DDhXRO4Dvgs8AqCqC4FiJwMzpj5srSdjIkRVy0XkIWAOMMa/TLgxnmNXFMZE1nhgN+CJzXuMCcQShTERIiKDqdqZ7lzgfv8OasZ4jiUKYyLAvxLps1TtabCDqgL2VGejMqZhLFEYExm3ATtUdb7/9jNAPxEZKSJLgL8BF4vIThFx7cq1xoCtHmuMMaYWdkVhjDEmJEsUxhhjQrJEYYwxJiRLFMYYY0KyRGGMMSYkSxTGGGNCskRhjDEmpP8PN0xCbeCf388AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plotBestFit(weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import  random\n",
    "def stocGradAscent1(dataMatrix,classLabels,numIter=200):\n",
    "    m,n=np.shape(dataMatrix)\n",
    "    weights=np.ones(n)\n",
    "    for i in range(numIter):\n",
    "        for j in range(m):\n",
    "            alpha=4.0/(5+i+j)+0.005\n",
    "            Index=random.randint(0,m-1)\n",
    "            h=sigmoid(sum(dataMatrix[Index]*weights))\n",
    "            error=classLabels[Index]-h\n",
    "            weights=weights+alpha*error*dataMatrix[Index]\n",
    "    return weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([12.67739518,  1.01691832, -1.62084593])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w1=stocGradAscent1(np.array(dataMat),labelMat)\n",
    "w1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3df3RddZnv8feTJv2VpKVtQqFpShOoKCJFqKnejgricAujomv0LljjyB0d27qUq3Pv3PprqXN1zVpOnRm9F0cpXBmZEXW8ozIuLQrquAAXlrbYUhRwoIGmLULSlv5IfyZ57h9nJzk5OWfnJDnn7O8+5/NaK6s5++wkz8lJ97O/z/eXuTsiIiKF1CUdgIiIhE2JQkREYilRiIhILCUKERGJpUQhIiKx6pMOoBxaWlp8+fLlSYchIpIaO3bs6HP31nzPVWWiWL58Odu3b086DBGR1DCz5wo9p9KTiIjEUqIQEZFYShQiIhJLiUJERGKVPVGY2Z1m9qKZPZ517K/MbL+Z7Yw+ri/wtWvN7Ckze9rMPlbuWEVEZLxKtCi+DqzNc/yL7n559LEl90kzmwH8A3AdcAlwk5ldUtZIRURknLInCnd/ADg0hS/tAp529z3ufgb4NnBDSYMTEZEJJdlH8SEzeywqTS3I83wb0JP1eF90LC8zW2dm281se29vb6ljFUmdniM93LLlFrru6OKWLbfQc6Rn4i8SySOpRPFV4ELgcuB54O/ynGN5jhXcPMPdb3f3Ve6+qrU17+RCkZrRc6SHlbetZPOOzWw7sI3NOzaz8raVShYyJYkkCnd/wd0H3X0IuINMmSnXPqA96/FS4EAl4hNJu02/3MTxM8c5O3QWgLNDZzl+5jibfrkp4cgkjRJJFGZ2ftbDdwCP5zltG7DCzDrMbCZwI/CDSsQnknZb928dSRLDzg6d5ZH9jyQUkaRZJYbHfgt4GLjYzPaZ2fuATWa228weA64G/iI6d4mZbQFw9wHgQ8BPgCeA77j7b8odr0g1WN22moa6hjHHGuoa6GrL13gXiWfVuGf2qlWrXIsCSi0b7qMYLj811DXQNLOJXRt20T6/feJvIDXHzHa4+6p8z2lmtkgVap/fzq4Nu1h/5Xq6lnSx/sr1ShIyZVW5zLhIteg50sOmX25i6/6trG5bzcY1G4u+2LfPb+fW628tc4RSC5QoRAKVWz7a+fud3L37brUMpOJUehIJlIa4SiiUKEQCpSGuEgolCpGAZC+7cWrgFPU2tjocyhBXLQ9SW9RHIRKI3D6J+rp6Bn2QeqtnwAdGhrhuXLMxqDjVd1L91KIQCURun8TA0AD1dfW8ovUVQQ1xVd9J7VGLQiQQhfok5tTPYev7tyYU1XjqO6k9alGIBCIty26kJU4pHSUKkUBsXLORpplNIxfhUPokcqUlTikdJQqRQKRl2Y1KxKlRVWHRooAiEhQtaJgMLQooIqmhUVXhUaIQkaBoVFV4lChEJCgaVRUeJQoRySupDuVyjqpSJ/nUqDNbRMZJukN5eB+OB/c+yJAPUWd1vH7Z6ye1H0e+76lO8sIS7cw2szvN7EUzezzr2BfM7Ekze8zMvm9m5xT42mejvbV3mpmu/CIVknSHcvv8djau2cjeI3t5su9Jdr2wi807NrPytpVTbgUk/ZrSrBKlp68Da3OO3Q9c6u6XAb8DPh7z9Ve7++WFMp2IxJtKuSWEDuVSX9hDeE1pVfZE4e4PAIdyjt3n7gPRw18BS8sdh0gtGi63bN6xmW0HthV9Vx5Ch3KpL+whvKa0CqEz+73AvQWec+A+M9thZuvivomZrTOz7Wa2vbe3t+RBiqTRp37+KV469dKk78pDWKaj1Bf2EF5TWiWaKMzsk8AAcHeBU9a4+xXAdcAHzewNhb6Xu9/u7qvcfVVra2sZopVy04iU0uo50sM3dn8DZ+yAlWLuykNYTqTUF/YQXlNaVWTUk5ktB37o7pdmHbsZ2ABc4+4nivgefwUcd/e/nehcjXpKH41IKb1bttzCV7Z9hSGGxhw3jPdc9h6aZzWzdf9WVretntZoonIaHv30yP5H6GrrCjbOahA36imR/SjMbC3wUeCNhZKEmTUCde5+LPr8WuCzFQxTKiiu4/LW629NOLp02rp/67gkAZlEcc9T93Di7Ingd6hrn9+u9z8AlRge+y3gYeBiM9tnZu8Dvgw0A/dHQ19vi85dYmZboi9dDDxkZruAR4AfufuPyx2vJEMjUkovX42/jjpWLFoxkiRAw0RlYmVvUbj7TXkOf63AuQeA66PP9wAryxiaBGR122p2/n7nmGQR8oiU4ZJIyKWbjWs2cvfuu8eV82bXz1ZSlkkJYdSTSKpGpEx1yGmlFeq8ff2y12uYqEyKlvCQYKSl4/KWLbewecfmca2f9VeuT0U9PaSBA7kts3df9m6+8dg3gm6pVau4zmwlCqk5PUd6+NTPP8W9T98LBtdddB2fu/pzRV+Quu7oYtuBbeOPL+li6/u3ljrcsgghKecmrHqrZ9AHmVE3g4GhAY18q7DgRj2JJKXnSA+v+uqrOHL6yMixu3bdxT1P3sPuD+wu6oKUtv6UfEIYTZQ70m0gWqxhYCjzr0a+hUN9FFIVip2st+mXmzh6+ui448dOHyt61E+a+lNClm+kWy51sodBLQpJvdwSRty8gK37t46bqQwwxFDRF6ThTuKkSzdpl69llittLbVqpRaFpN5kVhld3bYaw8Ydr6NuUhek4dLN1vdv5dbrb1WSmILcllm91WMY9XWZ+1e11MKhRCGpN5nJehvXbGTerHnjjjfPaq6JC1JI62nlDt/dsGoDD7/vYTZcuUFrMQVGpSdJvcl0LrfPb2f3B3bHjnoKeTLddGLLLdH9+vlfc8ejd/DylpdPe/e4qcrXqb566eqKxiAT0/BYSb1SzgsIaY5BqWPLN/9jWEivU5KR6FaoIuVWyuWjQ94uc7qxxY0yCul1SnhUepKqUKp5AeVYnLBUpazpxjbRKKOJvlfIJTkpLyUKkSylnkw3maG75Y4td5HAXHHfq5SvQ9JHpSdJtVKP4in1ZLpSlrKmG1t2iW7l4pXMmjGr6KGoIZfkpPzUopDUKsddbqkn002nXJSv1DPd2LJLdJNZ70n7hdQ2JQpJrXLtihfX3zHZOv0lrZew48COMTvNFVMuikuCpVr3aDL9OtWwvpVMnUpPklqVvsud7D4UPUd6uOfJe8ZtRzq3Ye6E5aLQSj3Vsr5VSBMO00SJQlIr31af5bzLnezFe9MvN3Hi7Ngt4Q3j7Re/fcJyUWilnlIOQU5KWjacCpEShaRWpe9yJ3vxzne+4zzR98SEP6vSSbAYaV/fKrRWWppUJFGY2Z1m9qKZPZ51bKGZ3W9m/xH9u6DA194cnfMfZnZzJeKVdKj0Xe5kL97TudhXS6knJKG10tKkUi2KrwNrc459DPiZu68AfhY9HsPMFgKfAVYDXcBnCiUUqU2VvMud7MV7Ohf7SiXBWqrZh9hKS4uKrfVkZsuBH7r7pdHjp4Cr3P15Mzsf+IW7X5zzNTdF56yPHm+OzvtW3M/SWk9SLpPdQjSELUcLCXldq3Kotdc7WaFuhbrY3Z8HiJLFuXnOaQOyb3H2RcfGMbN1wDqAZcuWlThUkYzJLhVSiS1Hp7q0RrmGF4dKG05NXejzKMbvMEOe7ckAd78duB0yLYpyBiUSiulMOgyhZl9MkivlGlMh7BWeRkkmihfM7Pys0tOLec7ZB1yV9Xgp8IsKxCaTpAXjklGoVfBH3/wjZtfPjn0vCi0SeHLgJD1Hesr+/hWT5LTGVH6nzg5ydnCI5tkNE59cAkn2UXwBOOjunzezjwEL3X1jztcsBHYAV0SHHgWudPdDcT9LfRSVpdpvcrru6GLbgW0Fn497L4bft2NnjjEwNDByvN7qaZ7VXPb3L9/+GA11Day/cv3IXX8x51SrwSHnwEsn6e7rZ0/v8cy/ff3s6e3nwJGT/Pc3v4xbrllRsp+XeB+FmX2LTMugxcz2kRnJ9HngO2b2PmAv8K7o3FXABnf/c3c/ZGafA4b/J3x2oiQhlVdrte6QFLN0eL73YrgF2D6/nX1H9nHo1Oh/qwEfqMj7V0zp64G9DyReHiu3Q/1n6O47zp7eTCLo7u2nu6+f7oP9nBkYndXfPKueztZGXrN8AZ2t7axZ0VKxGCuSKNz9pgJPXZPn3O3An2c9vhO4s0yhSQmEUOuuVRMtHQ7j34vcFmAxX1MOE60f1XOkh6f6nhr3dfVWn7ohrafODvLswUwSGG4V7OnLtBJeOjH6+uvrjGWL5tLZ0sQbL26ls6WRjpZGOlobaW2ahVm+btvyC70zW1JAC8YlJ3ckz8mBkzzR+wQDPlpKyn0vcluA+VTi/ctNcrnzTDb9chODPjju62bUzQhy4uHQkLM/plSUXeVfPG8WnS1NXP+q8+lsaaSztZHOliaWLphD/YzwFszQntkybYX6KO79k3v5xmPfSG0Hdwgd9JONoZj+oun0a5Ra3DyTQnGuXLySnRt2ljWuOIf7z0QJIJMMuqNk8OzBfk5nlYqaZtXTESWBzL9NdLY0srylkaZZ4d2jx/VRKFFISeT+h3/3Ze/muruvS20Hdwgd9FONYaJJfoU6iF/e8nLm1M8JZn5Bkh3Zp84O8tzBE5m+g+FSUZQYDueWihbOHZMMhpNDkqWiqVCikIpL+2iVEOIvVwwhJMFilDvOoSHn+aOnRstEvf08E32+/6WxpaLW5llRiaiJC7OSwtIFc2gIsFQ0FYmPepLak/YO7hDiL1cMaZmhXKo4XzpxZqRV0N03mhS6+8aWihpnzqCztYkrli3gnVcuzSSDliY6WsMsFVVSbb96KZu0d3CHEH+xMUylLyUtM5SLjfPU2UH2HjoxOpooSgR7+vo51H9m5LwZdcYFC+fS0dLI61e00NGSKRVd2NpIa3O6SkWVpNKTlEVayhuFhBB/MTGEEGelDJeKuqOWwTPD8w76jrPv8NhS0bnNs0b6Cjpbmkb6ENoXzq2aUlGpqY9CEhHyyqnFCCH+qXZMp6UvKJ8jJ8+O6Tfo7sv0HTx7sJ9TZ8eWijpaG+loaRozxHR5y9yKLW1RTZQoRKpUoSGkXUu62Pr+rQlEVJzTA4PsPXhiTN/BcFI4mFMqWrZwLssXzc0ML81qIZyrUlHGvHlw7Nj4483NcPRo0d9GndkiVaokfSklutDkGhpyfn/01EgieGZ4aYq+fvYdPsFQ1j1qS9MsOlsbufaVi8d0IrcvmMvMepWKYuV77+KOT4FaFCIpVpI+iri78iKuD9mlou6R5Sn6ebavn5NnR2dWz505Y+xcg6hctLylkXkqFU3dNN+/0W+jFoVIVarUUNfTA4P0HDox2iqIRhft6R1fKmpfMIeOlkZe17mIjtZGLoySw+J5KhWllRKFSBlUcvmPUg11deD3zYvoXtDGMwvb6F7Yxp5/fIQ9vflKRTPpaGnkza9YPGZW8rKFKhVVIyUKkRILfbOdo6fOjk4+6+3nmbdtpHthG90L2jg5c/bIeXPOnKLj6GletXQ+N1y+JGtUUSPz56hUVEuUKERKLIT9Oc4MDEUT0I6P2eNgT99x+o6PlorqDJaet4LOQ/t57d7ddBzaT+fhA3Qe2sfiY4eo86GYnyJBaG4uPBihRJQoREqsUst/uGdGFXX39vPMSDLIJIaeQ/lLRde8fPFIB/KFrY0sW9jIzIXnlP1CI2U0jZFpxVKiECmxUi//cWykVNQ/bnnrE2dGRxXNaciMKrp0yXzetjJTKlq+KNN3EFsqqsCFJlFlGv5bS5QoREpsog158hkuFXX3jU4+G56M1nf89Mh5dQZLF2TWKnrN8oVc2No4MhFtcfNs6urCGVUUwn4eQEXmGVS7xOZRmNnFwL9kHeoEPu3uX8o65yrg34Du6ND33P2zE31vzaOQpOVbemPpvKW8eOz0yFLWI62E3uP0HD7JYFataFHjzKxNb0bXKrpg0Vxm1c9I8JUVJ6g1qEo0z6DaBTmPwt2fAi4HMLMZwH7g+3lOfdDd31LJ2ESm49ipsxw+1sybl3yMjoZMMthw17N09/1mTKlodkMdyxc1csmSebzlsiWjw0xbmpg/N92jikLo0JfSCaX0dA3wjLs/l3QgIsU4OzhET7Ss9fBoouFyUe+x0VKRGSxdMIfOliZes3zhyBDTjtZGzp8XVqmolELYz0NKJ5REcSPwrQLPvc7MdgEHgL9099/kO8nM1gHrAJYtW1aWIKW2uDu9x06PzEbO7kTee+gEA1mlooVRqeiql7XSkbVw3bKFc5ndEH6pqNRC2M9DSifxtZ7MbCaZJPBKd38h57l5wJC7Hzez64H/7e4rJvqe6qOoXuXoID1+emDMkhTDLYTu3n76s0pFs+rrsvoNRvsOOlsaOWfuzOm+tKoSVB+FRj0VJehlxs3sBuCD7n5tEec+C6xy976485QoqtN0Lj7DpaLsReuGRxe9mFMqajtnTmYk0Zik0MiS+XPSWypK4GIZwn4eUrwgO7Oz3ESBspOZnQe84O5uZl1AHXCwksFJOCbqIHV3eo+fHjOaaDgx5JaKFsxtoLO1iTe8rDVrJdMmLlhUpaWiUg8RLSLxpGW7VZlYoonCzOYCfwiszzq2AcDdbwPeCXzAzAaAk8CNnnQTKE2qrMk93EFqPpsGb6N+qI0Gb+P+R9t46zMP0d3Xz/HTAyPnz6rPjCq6+LxmrnvVeSP7I3e2NLKgMaWlolDeU81NqCmJJgp3PwEsyjl2W9bnXwa+XOm4qkaK/zMPDA7Rc/jkmMlng30fYukpmOGjfzLOEHb2NOfMbeCPr2gb3eugNeWlokJS/J5KeoVQepIaNVwq6h6zPEWmI3nvwfGlorYF7Tx/8if0+3Octr34jBeYPesYD31gu2rfImWkRCFl1396YCQRDI8uGt785lhWqWhmfR0dixp52bnNrH3leWN2Q1sYlYp6jlwUdZDuoavtOnWQilSAEoWUxMDgEPsOn6S7r39kiYrhjuTfHz01cp4ZLJk/h87WRt5xRRudLY10RCOMlpwzhxkTlIoq1UEazDpFpVKBpahlAqH0L02BEoUUzd3pO35mzMJ1z0RLW+89dIKzg6OlovlzGuhsbWTNRS10tMwdWbhu+aLG4EcVhb7x0JSU+kKkxDN5Ke5fUqKoZlP8z3zizMC4ReuGS0fHTmWVimbUsbxlLhed28S1UanowmgRu4VpHVVE4OsUhXKBDvwOWEpLiaKaxfxnHhxy9h0+MdKB3J01K/n5I6fGnNt2zhw6Whp5x6vbov0NGrmwtamoUlEaBb1OkS7QkgAliirm7hzsPzPSKhjtTO5n78ETnBkc3eZy3ux6OlubeN2FizL9Bi2jpaI5M8MuFZVaEOsUpbiePaIaXoMAASzhUQ61toTHyTODWZ3HmYSQSQrHOZpTKrpg0dwxexx0RstTLGycicWt218BoXQgB7FOUTXsoVANr6GUAv99BL3WUzlUY6IYHHL2Hz7JM9FiddlDTA/klIqWzJ89soLp8OSzzpYm2haEWyoK4uKcE0+i6xRN9qIS4t174BfGigvxPcoy5bWeotVbW939mZzjl7n7YyWMUciUig6NlIrG7o/8XE6pqDkqFb22c9GY+QYdLeksFYXWgZy6dYpSPKKmZgSQDKaqYKIws/8CfAl40cwagP/q7tuip78OXFH+8Krfbw8c5f8+uGckKWSXihpmGBcsypSH3vSKc7kw2vCmo6WRRQGUikop6A7kfAK/O0zcvHlJRyAlFNei+ARwpbs/H63c+s9m9gl3/x5QPVeohJ04M8DDew7S0dLI2y5fkuk7iMpFbefMoX5GXdIhVkQQHciToTv4ePo9VJW4RFHv7s8DuPsjZnY18EMzWwrUYIGxPFYtX8jDH78m6TASt3HNRu7effe4PoqNazYmHVoyQpkvUQ7V8BpqTNzt6lEzu3D4QZQ0rgJuAF5Z5rikxrTPb2fXhl2sv3I9XUu6WH/l+nTPhJ6uo0czHb65H9VQ1qrUa5g3L9OhnvuhstikxbUoPkpOicndj5nZWuDjZY1KalLqOpBDUs0tkKlSebBk4loUdwF/bGYjycTMFgP/CLy13IGJyCRUcwtEEheXKK4EOoBfm9mbzOzDwCPAw8DqSgQnEqxCd+q1fAefTb+fqlKw9OTuh4ENUYL4KXAAeK2776tUcCLB0p16PP1+qkrBFoWZnWNmm4E/A9YC/wrca2ZvKmUAZvasme02s51mNm46tWX8HzN72sweMzPN3xCpVZXooFYn+DhxndmPAl8BPujuA8B9ZnY58BUze87dbyphHFe7e1+B564DVkQfq4GvotKXSLpNdcLiZDqop9rBr07wceISxRtyy0zuvhP4T2b2/vKGNcYNwD95ZlGqX0UtnfOH53iISApV4mKs8lfJFCw9xfVFuPsdJYzBybRWdpjZujzPtwE9WY/3RcfGMLN1ZrbdzLb39vaWMDyRKVIJQ6pECOtDrHH3K8iUmD5oZm/IeT7fciHjZoa7++3uvsrdV7W2tpYjTklCmi+2KmFIlUg8Ubj7gejfF4HvA7mL++wDsqfnLiUzAktqgS62IolLNFGYWaOZNQ9/DlwLPJ5z2g+A90Sjn14LHFH/hEiNqsT8DM0BGSfprVAXA9+PlsuuB77p7j82sw0A7n4bsAW4HngaOEFmuK5I6WjJ8Mqb6oikSrwfes/HSTRRuPseYGWe47dlfe7ABysZl9QYlbcqTxfjVEm8j0KkaqmEIVUi6dKTSLw0r4qqu2apEmpRSNjSuipqmof1iuRQohApB/V7SBVRohARkVhKFCIiEkuJQkREYilRiIhILCUKkXJIeg6FRl1JCWkehUg5JD18V6OupITUohAp1d2/7uKlSilRiJRqUp/u4osTYkINMaaAKFGISGWFmFBDjCkgShQiIhJLiUKkGiU96kqqihKFSNLKUR9P62KKEiQlCpFSmepdvOrjEjglCslPo0Amrxx38SH+/qf7txFiWSzEmAKiCXeSn+5yw1Sp33/cPuLT/dsIsfwVYkwBSaxFYWbtZvbvZvaEmf3GzD6c55yrzOyIme2MPj6dRKwiNUc3CpIlyRbFAPA/3P1RM2sGdpjZ/e7+25zzHnT3tyQQn4iIkGCLwt2fd/dHo8+PAU8AbUnFI5IY1cElcEF0ZpvZcuDVwNY8T7/OzHaZ2b1m9sqY77HOzLab2fbe3t4yRSpSBvk6wWuVBlEEKfFEYWZNwHeBj7h7bo/So8AF7r4SuBW4p9D3cffb3X2Vu69qbW0tX8C1QqNAkhXy77+csalvJEiJJgozayCTJO529+/lPu/uR939ePT5FqDBzFoqHGZt0oStZCX9+49LBknHJhWX5KgnA74GPOHuf1/gnPOi8zCzLjLxHqxclCJVaqISj5KBZEly1NMa4E+B3Wa2Mzr2CWAZgLvfBrwT+ICZDQAngRvda7mAK1IiKvHIJCSWKNz9IcAmOOfLwJcrE5GIiOSTeGe2SMlp5Ex6hdyJX8OUKKT6qKxSeaVKzuobCZIShYhMn5JzVVOiEKlFKvHIJGj1WJFapFKOTIJaFCJx1DEuokQhVaiUZRXV3kVUepIqpLJK5RXa0Eh9HlVBLQqRtAmxHKZhrVVNiUIkbVQOkwpTohARkVhKFCJxNN9ARIlCJJZq76UVYv+KTEiJQtKh2i4w1fZ6iqX+lVRSopB0qLYLzHRej8phUmGaRyGSNip7SYWpRSFj1WpJREQKUqKQsaqtxFNNlMQlIYkmCjNba2ZPmdnTZvaxPM/PMrN/iZ7fambLKx+llJQudlNXDUlc/SuplFiiMLMZwD8A1wGXADeZ2SU5p70POOzuFwFfBP6mslFKyU31YldtF5hqez3F0nDjVEqyRdEFPO3ue9z9DPBt4Iacc24A7oo+/1fgGjOzCsYooai2C0y1vR6pakkmijagJ+vxvuhY3nPcfQA4AizK983MbJ2ZbTez7b29vWUIV0SkNiWZKPK1DHwK52QOut/u7qvcfVVra+u0g6tZtVoSEZGCkkwU+4D2rMdLgQOFzjGzemA+cKgi0dUqlUTCpSQuCUkyUWwDVphZh5nNBG4EfpBzzg+Am6PP3wn83N3ztigkcMOjnQrRxW5iSuKSkMRmZrv7gJl9CPgJMAO4091/Y2afBba7+w+ArwH/bGZPk2lJ3JhUvDJNcaOalPtFgpboEh7uvgXYknPs01mfnwLeVem4RII0b17h7UbVqpAy0sxskbSohgl3kkpKFCIiEkuJQkREYilRSGVoaKdIamk/CqkMdbaKpJZaFCJpoVaZJEQtCpG0UKtMEqIWhQhonwyRGEoUIqA5CiIxlChAd5NSPvrbkiqgRAG6m6x15bxo629LqoAShYgu2iKxlChERCSWEoVIHM1REFGiEImluQsiShSAZrxKYdMdtaS/LakCmpkNumusdc3NhTcEmu6oJf1tSRVQi0LSoZzzEbQXtUisRFoUZvYF4K3AGeAZ4M/c/aU85z0LHAMGgQF3X1XJOCUgmo8gkpikWhT3A5e6+2XA74CPx5x7tbtfriQhIpKMRBKFu9/n7gPRw18BS5OIQ0REJhZCH8V7gXsLPOfAfWa2w8zWxX0TM1tnZtvNbHtvb2/Jg5QapVFLIuXrozCznwLn5Xnqk+7+b9E5nwQGgLsLfJs17n7AzM4F7jezJ939gXwnuvvtwO0Aq1at8mm/ABFQh7YIZUwU7v7muOfN7GbgLcA17p73wu7uB6J/XzSz7wNdQN5EIVUubgiriJRVIqUnM1sLfBR4m7ufKHBOo5k1D38OXAs8XrkoJSgawiqSmKT6KL4MNJMpJ+00s9sAzGyJmW2JzlkMPGRmu4BHgB+5+4+TCVdEpHYlMo/C3S8qcPwAcH30+R5gZSXjkioyb17hUpVaISKTEsKoJ5HS0wQ9kZJRohARkVhKFCIiEkuJQkREYilRiIhILCUKqU5aekOkZLRxkVQnDYEVKRm1KEREJJYShYiIxFKiEBGRWEoUIiISS4lCRERiWYGtIFLNzHqB55KOYxJagL6kg5gCxV05aYwZFHclTTfmC9y9Nd8TVZko0sbMtrv7qqTjmCzFXTlpjBkUdyWVM+aE4+cAAAQeSURBVGaVnkREJJYShYiIxFKiCMPtSQcwRYq7ctIYMyjuSipbzOqjEBGRWGpRiIhILCUKERGJpUQRCDP7nJk9ZmY7zew+M1uSdEzFMLMvmNmTUezfN7Nzko5pImb2LjP7jZkNmVnwQyDNbK2ZPWVmT5vZx5KOpxhmdqeZvWhmjycdS7HMrN3M/t3Mnoj+Pj6cdEzFMLPZZvaIme2K4v5fJf8Z6qMIg5nNc/ej0ef/DbjE3TckHNaEzOxa4OfuPmBmfwPg7h9NOKxYZvYKYAjYDPylu29POKSCzGwG8DvgD4F9wDbgJnf/baKBTcDM3gAcB/7J3S9NOp5imNn5wPnu/qiZNQM7gLen4HdtQKO7HzezBuAh4MPu/qtS/Qy1KAIxnCQijUAqMri73+fuA9HDXwFLk4ynGO7+hLs/lXQcReoCnnb3Pe5+Bvg2cEPCMU3I3R8ADiUdx2S4+/Pu/mj0+THgCaAt2agm5hnHo4cN0UdJrx9KFAExs782sx7gT4BPJx3PFLwXuDfpIKpMG9CT9XgfKbh4pZ2ZLQdeDWxNNpLimNkMM9sJvAjc7+4ljVuJooLM7Kdm9niejxsA3P2T7t4O3A18KNloR00Ud3TOJ4EBMrEnrpiYU8LyHEtFazOtzKwJ+C7wkZyWfrDcfdDdLyfTou8ys5KW+7QVagW5+5uLPPWbwI+Az5QxnKJNFLeZ3Qy8BbjGA+n0msTvOnT7gPasx0uBAwnFUvWiGv93gbvd/XtJxzNZ7v6Smf0CWAuUbCCBWhSBMLMVWQ/fBjyZVCyTYWZrgY8Cb3P3E0nHU4W2ASvMrMPMZgI3Aj9IOKaqFHUKfw14wt3/Pul4imVmrcOjDc1sDvBmSnz90KinQJjZd4GLyYzGeQ7Y4O77k41qYmb2NDALOBgd+lXoo7XM7B3ArUAr8BKw093/c7JRFWZm1wNfAmYAd7r7Xycc0oTM7FvAVWSWvn4B+Iy7fy3RoCZgZn8APAjsJvP/EOAT7r4luagmZmaXAXeR+fuoA77j7p8t6c9QohARkTgqPYmISCwlChERiaVEISIisZQoREQklhKFiIjEUqIQKYNoJdJuM1sYPV4QPb7AzH5sZi+Z2Q+TjlOkGEoUImXg7j3AV4HPR4c+D9zu7s8BXwD+NKnYRCZLiUKkfL4IvNbMPgL8AfB3AO7+M+BYkoGJTIbWehIpE3c/a2b/E/gxcG20TLhI6qhFIVJe1wHPA6nYvEckHyUKkTIxs8vJ7Ez3WuAvoh3URFJHiUKkDKKVSL9KZk+DvWQ6sP822ahEpkaJQqQ83g/sdff7o8dfAV5uZm80sweB/wdcY2b7zCzYlWtFQKvHiojIBNSiEBGRWEoUIiISS4lCRERiKVGIiEgsJQoREYmlRCEiIrGUKEREJNb/B3IiWzkvdfPCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plotBestFit(w1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def classifyVector(inX,weights):\n",
    "    prob=sigmoid(sum(inX*weights))\n",
    "    if prob>0.5:return 1.0\n",
    "    else:return 0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def colicTest(path=r'data\\{}.txt'):\n",
    "    frtrain=open(path.format('horseColicTraining'))\n",
    "    frtest=open(path.format('horseColicTest'))\n",
    "    trainingSet=[];trainLabels=[]\n",
    "    for line in frtrain.readlines():\n",
    "        currline=line.strip().split('\\t')\n",
    "        trainingSet.append([float(currline[i]) for i in range(21)])\n",
    "        trainLabels.append(float(currline[21]))\n",
    "    trainWeights=stocGradAscent1(np.array(trainingSet),trainLabels,1400)\n",
    "    errorCount=0;numTestVec=0.0\n",
    "    for line in frtest.readlines():\n",
    "        numTestVec+=1.0\n",
    "        currline=line.strip().split('\\t')\n",
    "        lineArr=[float(currline[i]) for i in range(21)]\n",
    "        if int(classifyVector(np.array(lineArr),trainWeights))!=int(currline[21]):\n",
    "            errorCount+=1\n",
    "    print('error:%f'%(errorCount/numTestVec))\n",
    "    return errorCount/numTestVec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "error:0.283582\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.2835820895522388"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colicTest()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def multiTest():\n",
    "    numTests = 3; errorSum=0.0\n",
    "    for k in range(numTests):\n",
    "        errorSum += colicTest()\n",
    "    print (\"after %d iterations the average error rate is: %f\" % (numTests, errorSum/float(numTests)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "error:0.268657\n",
      "error:0.402985\n",
      "error:0.373134\n",
      "after 3 iterations the average error rate is: 0.348259\n"
     ]
    }
   ],
   "source": [
    "multiTest()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
